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Abstract 

This study developed a method to assess group process in a 
collaborative problem-solving situation. Students in seven 
collaborative groups worked on a two-part math problem first 
individually, then in groups, and finally individually again. 
Groups engaging in behaviors that facilitated collaboration 
obtained higher group and individual accuracy scores on a 
challenging problem-set. High achieving students were 
influential in group problem-solving outcomes. Group scores did 
not reflect the individual achievement of low achieving 
students. Examining collaborative group process and outcomes 
offers a new direction in functional and contextualized 
assessment for school psychologists. 
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Assessing Group Process During Collaborative Problem-Solving 

Since the passage of the Education for All Handicapped 
Children Act (P.L. 94:142) over a quarter century ago, education 
for students with disabilities has been one of progressive 
transition from no services to segregated services to inclusion 
in regular classes (Nietupski, 1995). More recently, the Goals 
2000: Educate America initiatives passed by Congress (P.L. 103- 
227) challenged educators to serve increasingly diverse learners 
within regular education programs. Position statements issued by 
the National Association of School Psychologists (1993 a,b) 
support those reform efforts. Heterogeneous ability grouping and 
inclusive education reforms have obviated the need for 
assessments designed to determine special class placements or to 
classify within-child disabilities. Now the interest is less 
about why children fail and more about how or what they fail 
(Linn, 1993). Reschly (1988) predicted a revolution in school 
psychology fueled by educational reforms. New trends in 
assessment are shifting focus away from inferences about the 
psychological characteristics of children (e.g., cognitive 
abilities, motivation, personality) toward measures of the actual 
educational performance of children (e.g., achievement, 
functional skills, independence). Direct measures of student 
performance in realistic school contexts allow for low-inference, 
data-based educational decision-making and tighten the link 
between assessment and classroom practices. 



Collaborative Problem-Solving 4 



Collaborative learning, that is, students learning by 
interacting with each other rather than only with the teacher, 
has gained widespread acceptance as a strategy to manage academic 
heterogeneity in inclusive education settings (Cohen, 1994; 
Slavin, 1983 a,b, 1990). Theoretical support for collaborative 
group work comes from social constructivist (Vygotsky, 1981) and 
generative models of learning (Wittrock, 1989) which assume that 
interaction among children facilitates concept-learning and 
problem-solving. Pragmatically, the interpersonal skills needed 
to collaborate with diverse groups are recognized as important 
competencies for students preparing to enter the workforce 
(SCANS, 1991). The growing reliance on student collaboration in 
classroom instruction (Antil, Jenkins, Wayne, & Vadasy, 1998; 
Webb, 1997) demands an assessment of the effectiveness of those 
instructional strategies in terms of group and individual 
participation and achievement. The objectives of this study were 
to develop a method to assess group process and to examine 
achievement outcomes in a collaborative learning situation. 

Clark and colleagues (1996) described collaboration as a 
dialogue or discussion with its emphasis on group process and 
full participation of group members. When the goal of 
collaboration is learning or problem-solving, facilitative group 
processes may include exchanging ideas, giving and receiving help 
from others, clarifying strategies, resolving conflict, and 
encouraging others to participate. Collaborative learning is 
defined as students working together, without immediate teacher 
supervision, in a group small enough so that everyone can 
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collectively participate on a task. Several authors, notably 
Damon and Phelps (1989), have distinguished subtypes of 
collaborative learning models. Variously termed cooperative 
learning, peer collaboration, and peer tutoring, these subtypes 
differ somewhat based upon the demands for equality and mutuality 
in task engagement. Notwithstanding those distinctions, the term 
collaborative learning is used for of this study. 

Early studies of collaborative learning focused exclusively 
on heterogeneous group composition with respect to achievement 
rather than the interaction that occurs in those groups (see 
Webb, 1982). Bossert (1988), in a review of the research on 
collaborative education strategies, was critical of studies that 
employed a "black box approach" (p.233) in which a collaborative 
instructional method was compared to a non-collaborative method 
on some outcome measure. Distinctions between group versus 
individual learning settings are not the most important, rather 
the specific experiences of students are more important in 
predicting learning outcomes. Webb's longstanding research 
program (1980, 1982, 1989, 1993, 1995, 1997) demonstrates that 
the amount of student learning in group work depends on the 
quality of interaction. The important question concerns the ways 
in which various group collaboration processes produce positive 
effects . 

In a truly collaborative context, all individuals are 
actively engaged in working toward a solution to the problem 
(Damon & Phelps, 1989). Individuals work together by building on 
each other's ideas to construct understandings they did not have 
prior to the collaborative experience ( John-Steiner , Weber, & 
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Minnis, 1998; Tudge & Rogoff, 1989). Therefore, the performance 
of students working collaboratively with others would be a valid 
measure of individual competence when students are actively 
involved in learning how to solve the problem. On the other 
hand, when students use the group's resources to obtain a 
solution or an answer without trying to learn how to solve the 
problem (e.g., copying another student's work without trying to 
understand it), scores from a group assessment context will 
overestimate their individual competence. 

Test scores from group contexts may overestimate or 
underestimate students' performance in an individual setting. 
Group assessments that give students opportunities to collaborate 
may overestimate individual competence when students use 
resources in the group to solve problems that they would not be 
able to solve alone. Students with learning problems may perform 
better when working collaboratively, due to cognitive factors 
(e.g., greater intellectual resources available) and social 
variables (e.g., greater motivation). But negatively functioning 
groups may hinder performance more than individuals working 
alone . 

One solution to the problem of obtaining valid information 
about group and individual performance is to examine group 
processes. Studies of group processes in instructional settings 
are necessary for understanding how groups operate and the 
experiences of students in them. Cohen (1994) recommended 
observational studies that examine processes of interaction in 
relationship to outcome variables in order to analyze optimal 
features of interaction which are important for certain 
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outcomes. Group process data can reveal the extent and nature of 
individual student participation and individual student 
competence as well as the nature of the group's collaboration. 
This study is designed to illuminate how interaction in 
heterogeneous groups can prove effective in assisting low 
achieving students. 

In collaborative groups, it is expected that high achieving 
students will exert their influence to assist low achieving 
students. However, Dembo and McAuliffe's (1987) findings 
suggested that high-status students may maintain control in 
groups by discouraging the participation of lower-achieving 
students. Thus, groups may be "unfair" if they do not give 
students equal access to group resources. Inequalities in 
participation are troubling because participation may be linked 
to achievement gains (Fuchs, Fuchs, Hamlett, & Karns , 1998; Webb, 
Nemer, Chizhik, & Sugrue, 1998). 

Webb (1980) reported that group composition in terms of 
ability had important effects on the nature of the behavioral 
norms that developed in collaborative learning situations and 
these norms had an important impact on group process. Norms 
refer to implicit or explicit rules for acceptable behavior among 
group members which influence patterns of participation; for 
instance, who can ask or answer questions. If lower-achieving 
students have less access to social interaction they may be 
deprived of the benefits of giving or asking for help (Farivar & 
Webb, 1994). To understand group collaboration and individual 
participation, it is important to assess group norms for behavior 
so that an individual's participation can be compared. Focusing 
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on student capabilities and coping strategies in specific 
learning situations, rather than on pathology or disability, is a 
significant change in the perspective of assessment. 

Three practical implications of this review of the 
literature for school psychologists are: (1) the need to assess 
collaborative group processes; (2) the need to validate 
collaborative group and individual achievement outcomes; and (3) 
the need to teach students collaborative group process skills. 

The purposes of assessing group collaboration in this study were 
to measure a) group productivity, b) individual success after 
having had an opportunity to learn from group collaboration, and 
c) students' abilities to interact, collaborate with others, and 
function as members of a group. 

Method 

Participants : Fifth and sixth grade students attending a semi- 

private laboratory school on a college campus were recruited. 
Parental consent was obtained for fifty children to participate 
in the study. Eighteen of those children did not take part due 
to absences or lack of fit with the grouping parameters. The 
final sample consisted of 32 students assigned to eight groups. 
High and low math achieving students were categorized by median 
split on the mathematics subtests of the Metropolitan Achievement 
Test. Mixed gender, grade, and ability groups of four were 
constituted. Students assigned to one of the seven collaborative 
groups had not previously worked with other group members on 
problem-solving tasks. In addition, students rated their 
familiarity with each other to ensure each group was balanced 
along that interpersonal dimension. 
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Procedures ; A nonequivalent control group pre- and post-test 
design was used. Seven groups of four students worked to solve a 
challenging two-part math problem (Evered, 1997) both 
individually and collaboratively . As a control, four students 
worked individually but not collaboratively to solve the math 
problems. The problem-set is included in appendix A. All 
individually administered pre-tests were immediately followed by 
collaborative work for groups 1 through 7 on the first day of the 
study. Groups took between 5 and 10 minutes to solve both parts 
of the problem. No feedback was given to students about the 
accuracy of their work. Two weeks later, post-tests were 
administered to all students in the seven collaborative and 
single control groups. 

Discussion in the collaborative problem-solving groups was 
scored by the criteria set forth by Leach (1992) and Zola (1992). 
This scoring approach allows one to determine a norm of 
interaction in the group while evaluating an individual's 
participation. Examples of scoring categories include behaviors 
that facilitate group process (such as drawing another person 
into the discussion, asking a clarifying question, or 
communicating a problem-solving strategy) and behaviors that 
detract from group process (such as not paying attention, 
interrupting, or monopolizing). Three graduate students observed 
and videotaped collaborative group process during problem- 
solving. Facilitating or detracting group process behaviors were 
recorded for each student on a collaboration scoring tally 
sheet. Interobserver reliability coefficients for facilitating 
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behaviors, adjusted to account for chance agreements, were in the 
.80 range. Adequate interobserver reliability was not obtained 
in scoring behaviors that detract from group process so that 
category cannot be evaluated. Scores for facilitating discussion 
were calculated for each group as well as for individual members 
of each group as they worked to solve a problem set. Transcripts 
of collaborative problem-solving interactions and scoring 
protocol for groups one and seven, examples of groups scoring low 
and high in group process facilitation respectively, are 
contained in appendix B. On the transcript, statements are 
indexed by the number assigned to each group member (1 to 4) and 
by letter to indicate the sequence of the discussion. Statements 
are categorized and coded by letter on the collaborative scoring 
tally sheet. 

Results 

Insert Tables 1, 2, & 3 about here 
Information about the composition of each group in terms of 
gender, grade, and math achievement is contained in Table 1. 

Table 2 holds the problem-solving results for the seven 
collaborative groups. Individual pre-test scores for both parts 
of the problem and group scores are reported in columns 1 through 
4. In the next column, group facilitation scores are the sum of 
scores across all facilitating behavior categories. The final 
column lists the proportion of an individual group member's 
contribution to the collaboration — the student's individual score 
divided by the group facilitation score. Table 3 lists the 
individual pre- and post-test results for the seven collaborative 
groups and one control group for both parts of the problem. 
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Conclusions 

The findings suggest that high math achieving students were 
active participants and presumably influential in group 
interaction as evidenced by high individual contribution scores. 
Groups obtaining higher scores on behaviors that facilitated 
collaboration, that is, groups 4, 5, 6, and 7, achieved higher 
group accuracy scores on the problem-sets. Post-test findings 
showed that group outcome scores did not reflect individual 
achievement, particularly for the low math achieving students. 

In particular, this trend was apparent on the more difficult 
second part of the problem. Several low math achieving students 
from facilitative groups that obtained accurate group outcomes 
could not recall or recalculate the solutions on the post-test. 
However, more students from the facilitative groups obtained 
accurate post-test than did students in the collaborative groups 
scoring low in facilitation and the control group students who 
worked without group collaboration. A limitation to the study 
inheres in the design. It is not clear whether students who 
provided accurate answers on post-test following the 
collaborative problem-solving effort actually understood the math 
concepts involved or simply recalled answers from the group work. 

Discussion 

Notable about the findings was that positively interacting 
groups showed greater accuracy in group problem-solving and in 
individual achievement outcomes. In contrast, groups with low 
collaboration scores did not show individual problem-solving 
accuracy on post-test beyond that observed with the controls who 
did not have the opportunity for group collaboration. 




12 



Collaborative Problem-Solving 12 



These results are consistent with previous research findings 
emphasizing the importance of the quality of interaction among 
group members as critical to the outcomes of collaboration. In 
terms of a substantive contribution to the literature, this study 
provides a method for contextualized assessment for school 
psychologists in this era of educational reform. The 
collaboration scoring tally sheet can be used to simply count the 
frequency of behaviors or easily adapted for use as an interval 
time sampling protocol. Data obtained from collaborative groups 
provide a basis for practical, curriculum-based recommendations 
for school psychologists as they consult with teachers about 
students' needs and classroom practices. Information gained from 
group process and outcome assessment offers a conceptualization 
of collaboration in terms of facilitative interaction behaviors 
which may stimulate the formulation of interventions designed to 
optimize the participation and achievement of children working 
collaboratively classrooms. School psychologists are uniquely 
positioned and qualified to assist in promoting positive social 
interactions in groups. Effective collaborative group process 
data can be communicated to expand the skills of teachers without 
experience in inclusive education or those having specific 
difficulties in managing the diverse needs of students. 
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Table 1 

Group Composition 











Math 




Group 


Student 


Gender 


Grade 


Achievement 




1 


1 


F 


5 


High 






2 


M 


6 


Low 






3 


M 


5 


Low 






4 


F 


6 


High 




2 


1 


F 


5 


Low 






2 


M 


5 


High 






3 


M 


6 


Low 






4 


F 


6 


High 




3 


1 


F 


5 


High 






2 


F 


5 


Low 






3 


M 


6 


Low 






4 


M 


6 


High 




4 


1 


F 


6 


High 






2 


M 


6 


Low 






3 


M 


5 


High 






4 


F 


5 


Low 




5 


1 


M 


6 


Low 






2 


M 


5 


Low 






3 


F 


5 


High 






4 


F 


6 


High 




6 


1 


F 


5 


High 






2 


M 


5 


Low 






3 


M 


6 


High 






4 


F 


5 


Low 




7 


1 


M 


6 


High 






2 


F 


5 


Low 






3 


F 


5 


High 






4 


F 


6 


Low 




Control 


1 


F 


5 


High 






2 


M 


5 


Low 






3 


F 


6 


High 






4 


M 


6 


Low 
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Table 2 

Collaborative Problem-Solving Results 



Group 


Student 


Individual 
Pre-test 1 


Group 
Test 1 


Individual 
Pre-test 2 


Group 
Test 2 


Group 

Facilitation 

Score 


Proportion 

Individual 

Contribution 


1 






24* 




28 


9 






1 


NR 




44 






.33 




2 


NR 




26 






.00 




3 


15 




60 






.00 




4 


24 




30 






.67 


2 






24* 




47 


9 






1 


24 




18 






.11 




2 


26 




60 






.33 




3 


26 




30 






.04 




4 


24 




32 






.52 


3 






24* 




49 


10 






1 


24 




63 






.00 




2 


24 




NR 






.04 




3 


24 




32 






. 36 




4 


24 




60 






.61 


4 






24* 




46* 


17 






1 


24 




60 






.58 




2 


24 




90 






.23 




3 


NR 




NR 






.19 




4 


20 




29 






.00 


5 






24* 




46* 


18 






1 


24 




32 






.20 




2 


24 




60 






.15 




3 


24 




NR 






.24 




4 


24 




NR 






.41 


6 






24* 




46* 


19 






1 


24 




60 






.16 




2 


24 




32 






.07 




3 


24 




60 






.38 




4 


24 




36 






.38 


7 






24* 




46* 


23 






1 


24 




NR 






.23 




2 


20 




NR 






.17 




3 


24 




46 






.34 




4 


24 




26 






.26 



* Correct Answer 
NR No Response 
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Table 3 



Pre- and Post-Test Results 







Individual 


Group 


Individual 


Individual 


Group 


Individual 


Group Student 


Pre-test 1 


Test 1 


Post-test 1 


Pre-test 2 


Test 2 


Post-test 2 


1 


1 


NR 


24* 


24 


44 


28 


46 




2 


NR 




24 


26 




NR 




3 


15 




13 


60 




104 




4 


24 




24 


30 




32 


2 


1 


24 


24* 


24 


18 


47 


46 




2 


26 




24 


60 




46 




3 


26 




25 


30 




45 




4 


24 




AB 


32 




AB 


3 


1 


24 


24* 


24 


63 


49 


48 




2 


24 




24 


NR 




NR 




3 


24 




25 


32 




NR 




4 


24 




AB 


60 




AB 


4 


1 


24 


24* 


24 


60 


46* 


46 




2 


24 




24 


90 




46 




3 


NR 




AB 


NR 




AB 




4 


20 




24 


29 




60 


5 


1 


24 


24* 


24 


32 


46* 


46 




2 


24 




25 


60 




43 




3 


24 




24 


NR 




46 




4 


24 




24 


NR 




45 


6 


1 


24 


24* 


24 


60 


46* 


46 




2 


24 




46 


32 




46 




3 


24 




24 


60 




46 




4 


24 




NR 


36 




NR 


7 


1 


24 


24* 


24 


NR 


46* 


46 




2 


20 




24 


NR 




46 




3 


24 




AB 


46 




AB 




4 


24 




24 


26 




48 


Control 


1 


24 


XXX 


24 


30 


XXX 


45 




2 


24 




24 


NR 




30 




3 


24 




24 


79 




46 




4 


NR 




23 


56 




40 



* Correct Answer 
NR No Response 

VdY >sent 
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Post offices throughout the nation are required to display 
"wanted posters" of criminals. In an effort to economize, the 
postmaster general ordered every post office to use as few 
thumbtacks as possible to display their posters. The 
postmaster general’s directive included this illustration of a 
"tack-saving" display with 15 posters. Regulations require 
that each poster be fastened at all four corners as illustrated, 
and that each poster "overlap" another at least at one 
corner. 

Rearrange the posters so that each corner is fastened 

without using all 26 tacks. 

What would the minimum number of tacks required be? 

What arrangement would require the most tacks? 

How many would that be? 



(p 1 

yjvrsp 


> < 

u>AmT£0 


► i 


> 1 

u>ftr+TEO 


• < 


» 1 

uMnrEiD 


» i 


A k 1 


P i 

U>ft«4TS£ 


P 1 


P 1 






\f l 

tOanV<J 
<> < 


r < 

1 


10 Firt T£0 
^ 


9 
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Appendix B 

Illustrative Transcripts 
Collaboration Scoring Tally Sheets 
Groups 1 and 7 
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COLLABORATIVE PROBLEM SOLVING: GROUP 1 TRANSCRIPT 


* # 
A - 4: 


We still need 4 tacks. 


B - 1: 


Yeah, but it goes like that. 


C - 4: 


We don't need to go like that because it still overlaps - it has to 
overlap 1 corner - we still need that. 


D - 4: 


What did you guys do for 5 minutes? (directed at 2 & 3 ) 


E - 1: 


They guessed. 


F - 4: 


1,2 ,3 ,4 . . . I can count. 


G - 4: 


Nobody else did anything? 


H - 4: 


What's the minimum number of tacks required? 

What were the most tacks used? (reads questions from problem set) 
Anybody have any ideas? No? 


I - 1: 


If you put them is a square then you can use more tacks. 


J - 4: 


Let's try it - you want me to put a regular square - all right - here's 
a regular square. 


K - 4: 


Now — how many rows? (directed at 1) 


t - Is 


All you really need to do was this — (demonstrates on paper) - like 
that . 


M - 4: 


That's what I did. 


N - 4: 


This has 4 - that's too many - that's 15 there. 


0-4: 


Just do 3 down, this has 5 across - that uses less than 26 - it won't 
use more than 26 - You could do this - but no you could do this. 


P - 4: 


All right - you have 15 squares there, but this can still hold another 
(1, 2, 3 . . . ) 28 tacks - more than 26, less than 24. 


Q - 1: 


There wasn't enough - we could have just put a line through it. 


R - 4: 


Right (directed at 1 ) - no one right answer? 


S - 1: 


We're done aren't we? 


T - 4: 


I think we're done. 


U - 4: 


I ' m done . 



Statement Code 



# Seat 


Number 
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COLLABORATION 

GROUP 1 - Refer to Transcript 

Seat 1 Ila 
Seat 2 Mark 
Seat 3 Andy 
Seat 4 Erin 


SCORING TALLY SHEET 


BEHAVIORS THAT FACILITATE 


BEHAVIORS THAT DETRACT 


Communicating a strategy 
others 


Not paying attention/distracting 


51 

52 

53 

54 C.O 


51 

52 

53 

54 


Correctly applying a strategy 


Interrupting 


51 I , L 

52 

53 

54 P 


51 

52 

53 

54 


Recognizing errors 


Incorrect application or assumption 


51 Q 

52 

53 

54 N 


51 

52 

53 

54 


Drawing another into discussion 


Monopolizing 


51 

52 

53 

54 


51 

52 

53 

54 


Asking a clarifying question 


Making a personal attack 


51 

52 

53 

54 H.K 


51 E 

52 

53 

54 D.G 


Moving the discussion along 


Not contributing to group discussion 
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COLLABORATIVE PROBLEM SOLVING: GROUP 7 TRANSCRIPT 



* # 
A - 4: 

B - 2: 



C - 3: 
D - 4: 
E - 3: 
F - 2: 
G - 1: 
H - 4: 
I - 1: 
J - 4: 

K - 3: 

L - 4 
M - 1 
N - 3 
0-1 
P - 4 

Q - 1: 
R - 4: 
S - 3: 
T - 1: 
U - 2: 

V - 3: 
W - 4: 
X - 3: 

Y - 1: 
* o 

£RJC 



OK - how did you do yours? (directed at 2) 

Well, what I did is I took those 2 up here and I took off the bottom 
tacks so they'll just keep flopping around. 

Oh no, you have to keep all the tacks. 

Oh you can't do that? 

Each one has to have 4 tacks - I guess it's 24. 

OK - so 24. 

That's just the highest - I just kept adding to the highest. 

Well there are only 20 left to use. 

Oh no, but like you can move them around to add more tacks. 

You don't move them around to add tacks, you move them around to use 
less . 

If we try to make it as many tacks as possible and try to use . . . and 
figure out how many it was. 

What will require the most tacks? It will probably be this one. 

What did you get? (directed to 3) 

46. 

Oh all right, I get it, just so diagonal on the diagonal. 

The arrangement that would require the most tacks would be this one 
because — 26 . 

No, this one she made 4, no it's 26. 26 was the maximum. 

Oh, I thought it was ... cause all of them - OK so that's the highest, 
So just draw it on the back. 

No, you can just diagonal - you keep getting smaller and smaller. 
Where's the rest of it? 

The arrangement, do we have to draw that too? 

All you really have to do is draw these two. 

We can do these two here and down here. 

Yeah, and every other one. 

ement Code q o 

Number ® 
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COLLABORATION SCORING TALLY SHEET 

GROUP 7 - Refer to Transcript 

Seat l_Robert 

Seat 2_Katie 

Seat 3_Lia 

Seat 4_Megan 



BEHAVIORS THAT FACILITATE 

Communicating a strategy 
others 

S1_R , T 

S2_B 

S3_S 

S4 W 



BEHAVIORS THAT DETRACT 

Not paying attention/distracting 



51 

52 

53 

54 



Correctly applying a strategy 



S1_G,0,Q_ 
S2 



S3_E , K, X 
S4 P 



Interrupting 

51 

52 

53 

54 



Recognizing errors 



51 

52 



S3_C 

S4_H , J_ 



Incorrect application or assumption 



S1_G, I 

52 

53 

54 



Drawing another into discussion 

S1_M 

52 

53 



S4 A 



Monopolizing 

51 

52 

53 

54 



Asking a clarifying question 



SI 



S2_U 
S3_V 
S4 D 



Making a personal attack 



51 

52 

53 

54 



Moving the discussion along 



S1_Y 
S2_F 
S3 

© [ L 
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Not contributing to group discussion 



51 

52 

53 

54 
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